SMART High Precision: TREC 7

The Smart information retrieval project emphasizes completely automatic approaches to the understanding and retrieval of large quantities of text. We continue our work in TREC 7, concentrating on high precision retrieval. In particular, we present an in-depth analysis of our High-Precision Track results, including ooering evaluation approaches and measures for time dependent evaluation. We participated in the Query Track, making initial eeorts at analyzing query variability, one of the major obstacles for improving retrieval eeectiveness. In the Smart system, the vector-processing model of retrieval is used to transform both the available information requests as well as the stored documents into vectors of the form: D i = (w i1 ; w i2 ; : : :; w it) where D i represents a document (or query) text and w ik is the weight of term T k in document D i. A weight of zero is used for terms that are absent from a particular document, and positive weights characterize terms actually assigned. The assumption is that t terms in all are available for the representation of the information. The basic \tf*idf" weighting schemes used within SMART have been discussed many times. For TREC 7 we use the same basic weights and document length normalization as were developed at Cornell by Amit Singhal for TREC 4((3, 5]. Tests on various collections show that this indexing is reasonably collection independent and thus should be valid across a wide range of new collections. No human expertise in the subject matter is required for either the initial collection creation, or the actual query formulation. The same phrase strategy (and phrases) used in all previous TRECs (for example 2, 3, 4, 1]) are used for TREC 7. Any pair of adjacent non-stopwords is regarded as a potential phrase. The nal list of phrases is composed of those pairs of words occurring in 25 or more documents of the initial TREC 1 document set. Phrases are weighted with the same scheme as single terms. When the text of document D i is represented by a vector of the form (d i1 ; d i2 ; : : :; d it) and query Q j by the vector (q j1 ; q j2 ; : : :; q jt), a similarity (S) computation between the two items can conveniently be obtained as the inner product between corresponding weighted term vectors as follows: S(D i ; Q …